Method for determining a transformation of coordinates of different images of an object

ABSTRACT

Disclosed is a method for registering images of an object according to which a landmark that is common to the images is first identified, and the transformations in relation to the remaining degrees of freedom are determined by means of a position alignment that is not based on the landmark.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is the US National Stage of International ApplicationNo. PCT/EP2005/053201 filed Jul. 5, 2005 and claims the benefitsthereof. The International Application claims the benefits of Germanapplication No. 10 2004 032 914.1 filed Jul. 7, 2004, both of theapplications are incorporated by reference herein in their entirety.

FIELD OF THE INVENTION

The invention relates to a method for determining a transformation ofcoordinates of different images of an object, in which landmarksassigned to each other are searched for and the coordinate informationis computed with the aid of the image coordinates of the landmarks.

BACKGROUND OF THE INVENTION

These types of method are generally known in the area of medicaltechnology. They are especially used to align images which were obtainedusing different examination methods. The images involved can be bothvolume images and also projection images. For example the image of apatient that has been recorded with a computer tomography device can beplaced over a further image created using magnetic resonance tomography.The combination of a fluorescence image with an image created with theaid of a computer tomograph represents a further example.

The calculation of the necessary transformation of coordinates is alsoreferred to as registration. The presentation of the registered imagedata is also called fusion. The registration and the fusion can beundertaken with image data of the same or different modalities. Modalityin this case is understood as the way in which the data is obtained.Image data of the same modality has especially been recorded with thesame diagnosis device.

The image data of the same or different modalities can be registered theaid of orientation aids (=landmarks). These orientation aids, which arealso referred to as landmarks, can be easily identifiable areas of themapped object or additional markings (=fiducials) attached to theobject.

There are also methods which are oriented to the overall structure ofthe mapped object. These methods include methods with visual positionalignment and methods which compute the correlations between the voxelsof the images to be registered as well as methods which are oriented tothe surface of the mapped object. Voxels here are to be understood aspicture elements of a volume image.

For the registration of image data a certain number of degrees offreedom of a transformation matrix are defined, which map each imagecoordinate of the one image onto an assigned image coordinate of theother image. The one image is referred to below as the model image andthe other image as the reference image.

If the landmarks can be found in the image data, the transformationmatrix can be computed in a simple manner by solving a linear equationsystem. To this extent no error-prone or long-winded optimizationprocesses are necessary. In addition the transformation matrix can becalculated within a short time.

A disadvantage of the landmark-based calculation of the transformationmatrix is that the landmarks are frequently unable to be found in theimages. This leads to inaccuracies in the registration.

SUMMARY OF THE INVENTION

Using this prior art as its point of departure, the object of theinvention is to create an improved method for registration of images.

This object is achieved by a method with the features of the independentclaim. Advantageous embodiments and developments are specified in itsdependent claims.

The method involves initially searching for at least one pair oflandmarks assigned to each other. Subsequently the position of theobject contained in the different images is determined in relation tothe remaining degrees of freedom with the aid of a position alignmentnot based on the landmarks. The transformation of coordinates canfinally be computed with the information thus obtained.

In the method the information contained in the landmarks is effectivelyused for the position alignment. By comparison with conventionallandmark-based methods however the method does not fail even if thelandmarks cannot be completely found. Thus the method uses theinformation offered by the landmarks as far as possible but obtains themissing information using a method which is not dependent on landmarks.

The method is not restricted to position alignment between volumeimages, but can also be used for position alignment of projectionimages. It is also possible to match the projection of a volume imagewith a projection image.

Preferably the position alignment not based on the landmarks isundertaken by looking for an extreme value of a characteristic dimensionfor the correlation between the images. In this case thenon-landmark-based position alignment can be automated.

This part of the method can however also be performed if a user, on thebasis of images shown on a display, looks for a transformation ofcoordinates which leads to the greatest possible match.

BRIEF DESCRIPTION OF THE DRAWINGS

Further advantages and details can be taken from the description below,in which the exemplary embodiments of the invention are explained indetail on the basis of the enclosed drawing. The figures show:

FIG. 1 two images with different views of an object, whereby a positionalignment between the images is to be undertaken;

FIG. 2 a diagram with the transformations executed in a first exemplaryembodiment of the method for position alignment;

FIG. 3 a diagram with the transformations executed in a furtherexemplary embodiment of the invention for position alignment; and

FIG. 4 a diagram of a further embodiment of the invention, in which thepositions of the projection of a volume image and a projection image arealigned.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 shows an object 1 contained in different volume images 2 and 3.The volume images 2 and 3 are related to coordinate systems 4 and 5. Theposition of the object 1 in the volume images 2 and 3 can be differentin relation to the coordinate systems 4 and 5. If the volume images 2and 3 are then to be aligned in order to facilitate diagnosis for themedical personnel, a transformation of coordinates T must be foundthrough which the image coordinates of the object 1 in the volume image2 can be transformed into the image coordinates of the object 1 in thevolume image 3. In homogeneous coordinates the shape of thetransformation matrix is for example as follows:

$T = \begin{pmatrix}r_{11} & r_{12} & r_{13} & t_{x} \\r_{21} & r_{22} & r_{23} & t_{y} \\r_{31} & r_{32} & r_{33} & t_{z} \\0 & 0 & 0 & 1\end{pmatrix}$

The elements of the transformation matrix r₁₁ to r₃₃ describe a rotationof the object 1 in relation to the coordinate axes of the coordinatesystem 4 or 5. The elements of the transformation matrix t_(x) to t_(z)describes a translation that may be required along the coordinate axesof the coordinate system 4 or 5.

It should be noted that the volume images 2 and 3 are usually stored inDICOM format for medical applications. The DICOM format prescribes auniform scale for the volume images 2 and 3. It is thus not generallynecessary to undertake a stretching to align volume image 2 to volumeimage 3.

To compute the transformation of coordinate information about threedegrees of freedom of the translation and three degrees of freedom ofthe rotation is needed. Overall six degrees of freedom are thusavailable.

The transformation matrix T can now be determined as follows. As shownin FIG. 2, a landmark 6 can be contained both in the volume image 2 andalso in the volume image 3 respectively and can be easily identifiedboth in volume image 2 and also in volume image 3. The landmark 6 whichcan be found both in volume image 2 and also in volume image 3 can thenbe used directly to eliminate the degrees of freedom in relation to thetranslation. This means that the three translational elements t_(x) tot_(z) of the transformation matrix T are already known. As the methodproceeds only information regarding the three degrees of freedom of therotation still needs to be obtained, so that the remaining rotationalelements r₁₁ to r₃₃ can be computed.

The three rotational degrees of freedom are illustrated in FIG. 2 byrotations 7, 8 and 9 around the x-axis, y-axis and the z-axis of thecoordinate system 5. The rotations 7, 8 and 9 are those rotations bywhich the image data of the volume image 2 is rotated around the axes ofthe coordinate system 5.

The volume image 2 is also called the model volume image and the volumeimage 3 is called the reference volume image. The process by which thetransformation matrix T is determined is called registration. Finally,the application of the transformation matrix T to the image data of thevolume image 2 is called fusion.

Different registration methods which do not rely on landmarks can beused for determining the rotational elements of the transformationmatrix T.

For example the remaining registration after the landmark 6 has beenfound can be undertaken using an automatic registration method whichevaluates the voxels of the volume images 2 and 3. Typically a dimensionwhich is characteristic for the correlation between the image data ofthe volume images 2 and 3 is used in this case. This can for example bea coefficient of correlation which has been computed on the basis ofgrey values.

The registration in relation to the rotational elements of thetransformation matrix T can also be undertaken by a registration methodwhich is oriented to the surface of object 1. In addition registrationprocesses are considered, in which a user modifies the volume images 2and 3 displayed on the screen until such time as a sufficient match isachieved.

After the registration of the rotational elements of the transformationmatrix T a registration method which is not based on landmarks and whichmanages without using the pairs of landmarks 6 identified at the startcan again be applied to the image data. The renewed use of thenon-landmark-based registration method enables the previous result to beoptimized or verified.

FIG. 3 shows a further registration method, in which, after theidentification of two pairs of landmarks 10 and 11, the threetranslational elements t_(x) to t_(z) are known. The rotational elementsof the transformation matrix T are also largely defined since only onedegree of freedom remains in relation to rotation 7 around an axis ofrotation 12 defined by the landmark pairs 10 and 11. The angle ofrotation around the axis of rotation 12 still unknown after theidentification of the landmark pairs 10 and 11 is determined by aregistration method which is not landmark-based. This can be done bothmanually and automatically. Furthermore surface-oriented orvolume-oriented registration methods can be used. It is further possibleto improve the result of the registration by a further registrationmethod which is not based on landmarks and does not use the initiallyidentified pairs of landmarks 10 and 11.

The method described with reference to FIGS. 2 and 3 can also be appliedto projection images or to a combination of projection images withprojection of volume images. Such a registration method is shown in FIG.4. For the registration method shown in FIG. 4 the registration of avolume image 13 with a projection image 14 is shown. The transformationmatrix P generally has the form:

$P = {\begin{pmatrix}{f/d_{x}} & {f \star s} & u_{0} & 0 \\0 & {f/d_{y}} & v_{0} & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 1\end{pmatrix} \star \begin{pmatrix}r_{11} & r_{12} & r_{13} & t_{x} \\r_{21} & r_{22} & r_{23} & t_{y} \\r_{31} & r_{32} & r_{33} & t_{z} \\0 & 0 & 0 & 1\end{pmatrix}}$

11 degrees of freedom are to be taken into account in computing thetransformation matrix P: six intrinsic and five extrinsic. The extrinsicdegrees of freedom relate to the translation T1, T2 and T3 in thedirection of an x-axis, a y-axis and a z-axis and to the rotations R1,R2 and R3 around the x-axis, y-axis and the z-axis. The intrinsicdegrees of freedom are the degrees of freedom of the projection. Inparticular the quotients f/d_(x), f/d_(y), the product f*s as well as u₀and v₀ are designated as degrees of freedom. The variable f in this caseis the distance f between the one projection center 15 and theprojection image 14, d_(x) and d_(y) the pixel size in the projectionimage 14, s a stretching factor and u₀ and v₀ the coordinates of theso-called piercing point.

The task of registration is also simplified in this case if thepositional information communicated by a landmark 16 is evaluated first.This is because four pairs of landmarks are required in principle forknown intrinsic parameters in order to determine the position of theobject 1 in relation to the six extrinsic degrees of freedom. In thecase of unknown intrinsic parameters six pairs of landmarks arenecessary in order to define the eleven degrees of freedom. Even iffewer than four or fewer than six pairs of landmarks are identified, theidentified landmarks can be used to reduce the number of degrees offreedom, which makes the registration easier. The identification of thelandmark 16 enables the remaining registration process to be performedmore easily manually at a display unit and the optimization effort andthereby the computing time and the susceptibility to errors ofoptimizing registration methods which are based on voxels or surfacesfalls with the reduction of the degrees of freedom to be determined.

The outstanding feature of the methods described here is thus that anumber of landmarks are first identified in the images to be registered.The number of the landmarks is not sufficient however to form anequation system of which the solution is the transformation matrixsought. Following on from the identification of the landmarks theremaining degrees of freedom to be determined are defined byinteractive, semiautomatic or fully automatic registration methods.These registration methods can be a visual position alignment, a surfacescan matching or a fully automatic registration method, for example aregistration method based on the evaluation of voxels.

Finally, in a third method step, using a non-landmark-based registrationmethod, for example by an automatic registration based on voxels, thepreceding result can be optimized or verified without using thelandmarks identified at the start. A function can also be provided in adevice for executing the method which cancels the last method step.

1. A method for determining a transformation of coordinates of aplurality of different images of an object, comprising: assigning alandmark to the object; identifying the landmark in the different imagesof the object; determining elements in a transformation matrix based onthe landmark; determining remaining elements in the transformationmatrix with a position alignment not based on the landmark if thelandmark is not found or the remaining elements cannot be determined bythe landmark; and computing the transformation of coordinates using thetransformation matrix.
 2. The method as claimed in claim 1, wherein theposition alignment is oriented to an overall structure of the object. 3.The method as claimed in claim 1, wherein the position alignment isoriented to a volume of the object mapped in the images.
 4. The methodas claimed in claim 1, wherein the position alignment is oriented to asurface of the object mapped in the images.
 5. The method as claimed inclaim 1, wherein the transformation of coordinates is computed betweenvolume images.
 6. The method as claimed in claim 1, wherein thetransformation of coordinates is computed between projection images. 7.The method as claimed in claim 1, wherein the transformation ofcoordinates is computed between a projection of a volume image and aprojection image.
 8. The method as claimed in claim 1, wherein aposition of the object is aligned by the transformation of coordinates.9. The method as claimed in claim 8, wherein a characteristic measurefor a correlation between the images is optimized through the alignment.10. The method as claimed in claim 1, wherein the position alignment isperformed manually by a use matching positions of the images of theobject at a display unit.
 11. The method as claimed in claim 1, whereinthe position alignment is performed automatically by evaluating voxelsof the images of the object for a correlation between the images. 12.The method as claimed in claim 1, wherein the landmark is identifiedautomatically.
 13. The method as claimed in claim 1, wherein thelandmarks is identified by a user at a display unit.
 14. The method asclaimed in claim 1, wherein the object is a patient.
 15. The method asclaimed in claim 1, wherein the transformation of coordinates isoptimized or verified by determining all elements in the transformationmatrix with the position alignment not based on the landmark.
 16. Adevice for determining a transformation of coordinates of a plurality ofdifferent images of an object having a landmark, the landmark beingidentified in the different images, comprising: a display unit thatdisplays the different images; and a computing unit that computes thetransformation of coordinates by determining a transformation matrix,wherein the transformation matrix comprises elements based on thelandmark and elements determined by a position alignment not based onthe landmark if the landmark is not found or the remaining elementscannot be determined by the landmark.